Experiments in Parameter Learning Using Temporal Differences

In this paper we discuss the problem of automatically learning evaluation function parameters in a chess program. In particular, we describe some experiments in which our chess program KnightCap learnt the parameters of its evaluation function using a combination of Temporal Difference learning and on-line play on FICS and ICC. KnightCap is freely available on the web from http://wwwsyseng.anu.edu.au/lsg. The main success we report is that KnightCap went from a (blitz) rating of 1650 to a rating of 2150 in just 3 days and 308 games. We discuss the details of our learning algorithm, details of KnightCap, and some of the principal reasons for KnightCap’s rapid improvement.